AR-based real-time communication systems or methods are known. For example, Apple Photobooth™ allows users to create photos or videos of themselves in a virtual environment. Sometimes, the virtual environment can be a moving environment. Google Hangout™, an audio and video conference platform, allows users to select desired individualized background during a video conference session and allows users to wear exchangeable virtual items such as hats, glasses and mustache. However, such existing systems are crude and primitive from a visual perspective. Often there are obvious and sometimes significant gaps and imperfections at locations where a real life object (a user on Google Hangout™) interfaces with a virtual item (e.g., a virtual background and a wearable decorative item). Such defects are more pronounced where a virtual environment includes moving elements.
Under some situation, professional photographers or film makers use green screens for initial image or video capturing to minimize defects. Extensive editing is needed at a later time to replace the monotone background to a virtual environment. This is very time consuming and expensive process. During video conferences (such as Google Hangout™), people with different language may also get lost and result in ineffective communication.
What is needed in the art are improved systems and methods for real-time processing image and other signals (such as audio signals) for AR-based communications.